1. Technical Field
The present invention relates to an image quality enhancement method of an image, more specifically, an image quality enhancing apparatus, an image display apparatus, an image quality enhancing method, and a computer readable storage medium, which execute a learning-type image quality enhancing method utilizing sparse expression.
2. Related Art
An image is sometimes enlarged when handling an image such as when displaying an image, etc. When the pixel count is simply increased to enlarge the image, a problem that the resolution decreases and the image quality degrades occurs. In order to solve this problem, a technique to enhance an image quality of an image is being developed. For example, a technique of a learning-type super-resolution is being developed. In the learning-type super-resolution technique, a dictionary which has learned in advance a correspondence relationship between a low-quality image and a high-quality image is created, and the image quality of an image is enhanced by extracting a high-quality image corresponding to an actual low-quality image from the dictionary.
As one method of the learning-type super-resolution, Non-Patent Document 1 discloses a learning-type image quality enhancing method which utilizes a sparse expression. In Non-Patent Document 1, the image quality enhancement is carried out by the procedures as the following. A high frequency component of the luminance is extracted from a small region (referred to as a patch below) with each pixel in the image at the center. The high frequency component here is a component that is changing with a high frequency of higher than or equal to a predetermined spatial frequency within the luminance distribution in a patch. The extracted high frequency component of the luminance indicates the luminance distribution in the patch. The degradation of an image quality occurs by a downsampling or enlarging, and this degradation of the image quality occurs in a high frequency component of the luminance. The high frequency component of the luminance expresses a feature of the image, and thus the extracted high frequency component of the luminance is the feature quantity of the image. A feature quantity of any image can be expressed by a combination of a plurality of predetermined fundamental feature quantities. The feature quantity is expressed by vectors, and the plurality of predetermined fundamental feature quantities are called base vectors. The feature quantity of any image is expressed by a linear sum of a plurality of base vectors. The one-on-one correspondence relationship between a low-image-quality base vector and a high-image-quality base vector is learned in advance, and a dictionary data in which the learned contents are recorded is created. Based on the feature quantity of the image, a coefficient of each base vector for expressing the feature quantity with the linear sum of a plurality of low-image-quality base vectors is determined. At this time, the coefficient is determined such that the number of the base vectors with non-zero coefficients is as small as possible. To make the number of the base vectors with non-zero coefficients as small as possible is called a sparse expression. By multiplying each of the high-image-quality base vectors by the coefficients of the corresponding low-image-quality base vectors, and by calculating the sum of the high-image-quality base vectors multiplied by the coefficients, a high frequency component of the luminance is reconstructed. The reconstructed high frequency component is the high frequency component of the luminance of a high-quality image. By combining the low frequency component of the luminance and the reconstructed high frequency component of the luminance, a high-quality image is generated.
The feature quantity of an image is to be expressed with a linear sum of the base vectors with non-zero coefficients. The coefficients of almost all base vectors become zero by using the sparse expression, so the operation amount decreases. Also, by the sparse expression, the feature quantity of the image is expressed by the base vector that is the most influential, and thus a base vector selected does not change even if the feature quantity of the image varied in some degree due to the noise. Therefore, a robust result can be obtained for the noise. Then, it becomes important how the sparse expression is realized.
A set of a plurality of base vectors is D, a coefficient matrix consisting of the coefficients of the base vectors is α, and the feature quantity of the image is y. With m and n as natural numbers, D is a matrix with m lines and n columns, α is a matrix with n lines and one column, and y is a matrix with m lines and one column. Since the feature quantity of the image is expressed with the linear sum of a plurality of base vectors, Dα=y holds. If m=n, a solution of α can be uniquely obtained. However, there is no guarantee that the solution of the α obtained here is a sparse solution. Also, if m<n, the solution where L2 norm of α becomes minimum can be obtained by a solution method using a general inverse matrix. However, also in this case, there is no guarantee that a sparse solution of a can be obtained.
In Non-Patent Document 1, Dα=y is not considered a strict condition, and α that is more sparse is selected from among the α's where Dα≈y. Specifically, an operation to solve the conditional equation expressed by Equation (1) below is performed.
                    [                  Eq          ⁢                                          ⁢          1                ]                                                                                  min            α                    ⁢                                                                                    D                  ⁢                                                                          ⁢                  α                                -                y                                                    2            2                          +                  λ          ⁢                                                  α                                      1                                              (        1        )            
The ∥·∥1 is L1 norm, ∥·∥2 is L2 norm in Equation (1), and λ is a parameter referred to as a sparse constraint term. The meaning of Equation (1) is to solve for α such that ∥α∥1 is preferably small and the difference between the linear sum of the base vectors and the feature quantity is small. The α where ∥α∥1 becomes as small as possible is a sparse solution. A differentiation of L1 norm is generally difficult, and solving Equation (1) analytically is difficult. In Non-Patent Document 1, a solution of Equation (1) is obtained by letting the solution converge by the iteration method.